Reallocation of electronic resources using a predictive model of attribution

ABSTRACT

Systems and methods for predictive modeling of attribution are described. Systems and methods may include receiving one or more inputs; processing the one or more inputs using a general linear model; and providing predicted online and offline campaign impact.

INCORPORATION BY REFERENCE TO RELATED APPLICATIONS

This application claims benefit under 35 U.S.C. § 119(e) from U.S.Provisional Application No. 62/025,162, filed on Jul. 16, 2014 and U.S.Provisional Application No. 62/025,158, filed on Jul. 16, 2014. Thedisclosures of each of the applications cited in this paragraph areincorporated herein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates to systems and methods for marketingcampaigns, and, more specifically, to systems and methods for improvingspeed of online and offline attribution.

BACKGROUND OF THE INVENTION

Targeted marketing is a commonly used tool for improving return oninvestment for advertising expenditures. In general, the more accuratethe targeting is to consumers, the more benefit is received from theadvertising campaign.

Measurement of the effectiveness of advertising campaigns providesfeedback that can be used to determine whether the advertising campaignhas been effective. The current industry technology uses stratifiedsample groups of campaign prospects separated into a treated and controlgroup to measure effectiveness of a campaign incrementally. Thesedeterminations are made on a monthly basis. Existing technology does notoptimize campaign return on investment because it does not utilize realtime data to adjust for optimization. In addition, current industrytechnology targets based on cookies or sites and not based on emailaddress.

Needs exist for improved systems and methods for improved systems andmethods for marketing campaigns.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate preferred embodiments of theinvention and together with the detailed description serve to explainthe principles of the invention. In the drawings:

FIG. 1 shows an exemplary system for predictive modeling of attribution.

FIG. 2 shows an exemplary system for computational aspects of predictivemodeling of attribution.

FIG. 3 shows an exemplary flow diagram for predictive modeling ofattribution.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Systems and methods are described for using various tools and proceduresfor optimizing targeted advertising. In certain embodiments, the toolsand procedures may be used in conjunction with improved attribution. Theexamples described herein relate to marketing campaigns, including emailand Internet based advertising campaigns, for illustrative purposesonly. The systems and methods described herein may be used for manydifferent industries and purposes, including any type of marketingcampaigns and/or other industries completely. In particular, the systemsand methods may be used for any industry or purpose where customizedcustomer identification is needed. For multi-step processes or methods,steps may be performed by one or more different parties, servers,processors, etc.

Certain embodiments may provide systems and methods for targetedadvertising. A set of information may be accessed from one or moredatabases. The information may include various types of information,including, but not limited to, real time campaign information, audienceprofiles, and attribution data. A model may be accessed or created. Themodel may be a general linear model for determining factors forpredicting results of an advertising campaign. The general linear modelmay be used to project online and offline impacts of marketingcampaigns.

An email channel may be any communication sent electronically to anelectronic address, i.e., sent via email. In certain embodiments, anemail channel may refer to sending of third party advertisements throughemail.

In general, inventory may be a term for a unit of advertising space,such as a magazine page, television airtime, direct mail message, emailmessages, text messages, telephone calls, etc. Advertising inventory maybe advertisements a publisher has available to sell to an advertiser. Incertain embodiments, advertising inventory may refer to a number ofemail advertisements being bought and/or sold. The terms inventory andadvertising inventory may be used interchangeably. For email marketingcampaigns, advertising inventory is typically an email message.

A publisher may be an entity that sells advertising inventory, such asthose produced by the systems and methods herein, to their emailsubscriber database. An advertiser may be a buyer of publisher emailinventory. Examples of advertisers may include various retailers. Amarketplace may allow advertisers and publishers to buy and selladvertising inventory. Marketplaces, also called exchanges or networks,may be used to sell display, video, and mobile inventory. In certainembodiments, a marketplace may be an email exchange/email marketplace.An email exchange may be a type of marketplace that facilitates buyingand/or selling of inventory between advertisers and publishers. Thisinventory may be characterized based on customer attributes used inmarketing campaigns. Therefore, an email exchange may have inventorythat can be queried by each advertiser. This may increase efficiency ofadvertisers when purchasing inventory. A private network may be amarketplace that has more control and requirements for participation byboth advertisers and publishers.

An individual record/prospect may be at least one identifier of atarget. In certain embodiments, the individual record/prospect may beidentified by a record identification mechanism, such as a specificemail address (individual or household) that receives an email message.

An audience may be a group of records, which may be purchased asinventory. In certain embodiments, an audience may be a group of recordsselected from publisher databases of available records. The subset ofselected records may adhere to a predetermined set of criteria, such ascommon age range, common shopping habits, and/or similar lifestylesituation (i.e., stay at home mother). Advertisers generally select thepredetermined set of criteria when they are making an inventorypurchase.

Although not required, the systems and methods are described in thegeneral context of computer program instructions executed by one or morecomputing devices that can take the form of a traditionalserver/desktop/laptop; mobile device such as a smartphone or tablet;etc. Computing devices typically include one or more processors coupledto data storage for computer program modules and data. Key technologiesinclude, but are not limited to, the multi-industry standards ofMicrosoft and Linux/Unix based Operating Systems; databases such as SQLServer, Oracle, NOSQL, and DB2; Business Analytic/Intelligence toolssuch as SPSS, Cognos, SAS, etc.; development tools such as Java,.NETFramework (VB.NET, ASP.NET, AJAX.NET, etc.); and other e-Commerceproducts, computer languages, and development tools. Such programmodules generally include computer program instructions such asroutines, programs, objects, components, etc., for execution by the oneor more processors to perform particular tasks, utilize data, datastructures, and/or implement particular abstract data types. While thesystems, methods, and apparatus are described in the foregoing context,acts and operations described hereinafter may also be implemented inhardware.

FIG. 1 shows an exemplary system 100 for predictive modeling of onlineand offline attribution according to one embodiment. In this exemplaryimplementation, system 100 may include one or more servers/computingdevices 102 (e.g., server 1, server 2, . . . , server n) operativelycoupled over network 104 to one or more client computing devices 106-1to 106-n, which may include one or more consumer computing devices, oneor more provider computing devices, one or more remote access devices,etc. The one or more servers/computing devices 102 may also beoperatively connected, such as over a network, to one or more thirdparty servers/databases 114 (e.g., database 1, database 2, . . . ,database n). The one or more servers/computing devices 102 may also beoperatively connected, such as over a network, to one or more systemdatabases 116 (e.g., database 1, database 2, . . . , database n).Various devices may be connected to the system, including, but notlimited to, client computing devices, consumer computing devices,provider computing devices, remote access devices, etc. This system mayreceive inputs 118 and outputs 120 from the various computing devices,servers and databases.

Server/computing device 102 may represent, for example, any one or moreof a server, a general-purpose computing device such as a server, apersonal computer (PC), a laptop, a smart phone, a tablet, and/or so on.Networks 104 represent, for example, any combination of the Internet,local area network(s) such as an intranet, wide area network(s),cellular networks, WIFI networks, and/or so on. Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, etc. Client computing devices 106, which may include at leastone processor, represent a set of arbitrary computing devices executingapplication(s) that respectively send data inputs to server/computingdevice 102 and/or receive data outputs from server/computing device 102.Such computing devices include, for example, one or more of desktopcomputers, laptops, mobile computing devices (e.g., tablets, smartphones, human wearable device), server computers, and/or so on. In thisimplementation, the input data comprises, for example, real timecampaign data, audience profile, attribution data, and/or so on, forprocessing with server/computing device 102. In one implementation, thedata outputs include, for example, emails, templates, forms, and/or soon. Embodiments of the present invention may also be used forcollaborative projects with multiple users logging in and performingvarious operations on a data project from various locations. Embodimentsof the present invention may be web-based, smart phone-based and/ortablet-based or human wearable device based.

In this exemplary implementation, server/computing device 102 includesat least one processor coupled to a system memory. System memory mayinclude computer program. modules and program data.

In this exemplary implementation, server/computing device 102 includesat least one processor 202 coupled to a system memory 204, as shown inFIG. 2. System memory 204 may include computer program modules 206 andprogram data 208. In this implementation program modules 206 may includedata module 210, model module 212, analysis module 214, and otherprogram modules 216 such as an operating system, device drivers, etc.Each program module 210 through 216 may include a respective set ofcomputer-program instructions executable by processor(s) 202. This isone example of a set of program modules and other numbers andarrangements of program modules are contemplated as a function of theparticular arbitrary design and/or architecture of server/computingdevice 102 and/or system 100 (FIG. 1). Additionally, although shown on asingle server/computing device 102, the operations associated withrespective computer-program instructions in the program modules 206could be distributed across multiple computing devices. Program data 208may include campaign data 220, audience data 222, attribution data 224,and other program data 226 such as data input(s), third party data,and/or others.

As shown in FIG. 3, certain embodiments may take one or more types ofinformation, passes them through one or more models, and projects onlineand offline campaign impacts.

A system 301 may include one or more input sources 303 that provide oneor more items of data. Data may be accessed from and/or provided by oneor more sources. In certain embodiments, input sources 303 may include,but are not limited to, real time campaign data 305, audience profiles307, and/or attribution data 309. Items of data may be stored locally orremotely. Items of data may be stored in one or multiple databases.

Real time campaign data 305 may include one or more of the following:

-   -   opens (action of an email recipient opening an email, which may        mean clicking “show images”);    -   clicks (action of an email recipient clicking on email content,        which sends them to a landing page in a web browser);    -   landing page actions (action an email recipient complaining on        an email, which may include indicating complain in a mail client        program);    -   complaints (action of an email recipient unsubscribing on an        email, which may include clicking unsubscribe to prevent further        emails from the sender or advertiser);    -   unsubscribes (action of an email recipient unsubscribing on an        email, which may include clicking unsubscribe to prevent further        emails from the sender or advertiser);    -   metrics rates (calculated metrics that indicate (performance of        a email campaign): Metrics may be calculations or computed        values that are used to measure campaign performance. For        example, open rate (the percentage of opens over possible opens)        indicate the engagement levels of the email campaigns.        Additional metrics may include, but are not limited to,        click-through rate (the rate of clicks to possible clicks) s        well as additional advertiser specific performance measurements.        These metrics can also be considered over time, for example, if        the open rate of a campaign starts at X and increases by a        margin in the first 5 hours of the campaign, this increase can        be used as an input as independent variable in predicting        subsequent action;    -   rate of change of metric rates (additional calculated metrics):        Velocity of a metric may be a calculation of the rate of change        of a metric X. This calculation may yield a second metric, Y,        which represents a new data point around which decisions can be        made. If two metrics are comparable, the one that is moving in a        directionally positive manner may be of greater use in        computations; and    -   datetime (date, time, seasonality, as well as other time-based        indications).

Audience profiles 307 may include individual and household leveldemographics from both self-reported sources and third party vendors,digital shopping behavior across other marketing campaigns, and offlineshopping behavior sourced from catalogues, loyalty cards, retail stores,etc. Audience profiles 307 may include one or more of the following:

-   -   demographics explicit information on the email record individual        such as, but not limited to, age, gender, income, marital        status, etc.);    -   geographic (explicit information on the email record such as,        but not limited to, postal address, zip code, state, etc.);    -   online sales (previous online behavior of an email recipient,        such as, but not limited to, signing up for one or more        services, purchasing one or more products, etc.);    -   offline sales (previous offline behavior of an email recipient,        such as, but not limited to, signing up for one or more        services, buying one or more products, etc. This may be based on        offline SKU level data from retailers, catalogues, loyalty card        activity, etc., and may be matched to email prospects based on        various identifiers, such as name, postal address, etc.);    -   psychographic (description of personality, values, opinions,        attitudes, interests, lifestyles, etc. that allow advertisers to        customize content to improve response); and    -   purchase intent data. Purchase intent may be determined based on        comparisons between the actions on a specific advertisement        compared to a population average. For example, if females age        24-35 click on skin care advertisements at a rate of three times        the national average, they may have a three times purchase        intent multiplier.

Attribution data 309 may include measurements of the impact of anadvertising campaign. Attribution may be a methodology behind measuringthe impact of advertising campaigns. Attribution may be a process toidentify a set of user actions (“events”) that contribute in some mannerto a desired outcome, and then assigning a value to each of theseevents. In certain embodiments,attribution may determine a total impactof email campaigns not only based on activity online, hut also whetherthe advertisement contributes to offline activity, such as when theemail recipient make a purchase in a brick and mortar store.

To measure campaign impact, an experiment may be performed in which theonly difference between two groups of record sets is that one receivesan advertisement (treatment group) and one does not (control group).These groups are created based on a stratified sampling process, whichensures that the attributes or characteristics of each group areproportional to each other. After a campaign is executed, the treatmentgroup and the control group are compared to the new customer fileprovided by the advertiser. There may be specific criteria to determinea “match”. These criteria may include, but are not limited to, a timerange (i.e., purchased within 30 days of receiving the advertisement)and a key d (i.e., email, or name and postal address).

With this match information, the new customer rate for both thetreatment group and the control group are compared. The differencebetween these treatment group and the control group customer rates maybe the incremental new customer rate of a campaign. The product of thetreatment population and the incremental customer rate may be theincremental customers the campaign generated. Using this information, inaddition to the cost of the advertising, may provide a true return oninvestment of the media spend.

In certain embodiments the above process may be executed in real timeand/or close to real time.

Certain embodiments may allow for continuously matching the treatmentand control files to an advertiser's customer file, and computing theincremental customer rate and the cost per new customer on a continuousand/or near continuous basis across campaigns. If multiple campaigns arelaunched simultaneous for a specific advertiser, certain embodiments mayallow for measuring relative performance of the multiple campaigns andshifting media spend to a better performing campaign. Additionally,certain embodiments may use this modeling information to predict a finalreturn on investment target for a particular campaign.

Attribution data 309 may be based on stratified micro-sampling.Micro-sampling may consider both control groups and treated groups.Control groups may be groups of email recipients that do not receive anadvertisement. Treated groups may be groups of email recipients that doreceive an advertisement. Attribution data 309 may allow measurement inreal or near real time of an incremental lift of a campaign. Incrementallift may be a measured impact from campaigns by comparing response ratesof treated and control groups. For example, a determination may be madeas to whether a response to an advertisement by a treated group isgreater than the response by a control group, which is not treated. Aprecise significance test may be performed in real time. Significancetests are well-known for determination of whether a value is considered“significant” (i.e., is not simply due to chance). The probability thata variable would assume a value greater than or equal to the observedvalue strictly by chance may also be determined by a significance test.

Attribution data 309 may include one or more of the following:

-   -   treatment group/treated prospects records: records that will        receive an advertisement;    -   control group/control prospects records: records that will not        receive an advertisement;    -   advertiser customer data/new customer file: sales information        provided by an advertiser;    -   customer matches: matches between a treatment group or control        group and the new customer file on specific criteria based on        the advertiser;    -   treatment new customers: number of new customers that match the        treatment group;    -   control new customers: number of new customers that match the        control group;    -   treatment new customer rate: percent of new customers that were        treated over the total treatment prospects;    -   control new customer rate: percent of new customers that were        not treated over the total control treatment prospects;    -   incremental new customer rate/incremental customer rate:        difference between the treatment new customer rate and the        control new customer rate; and    -   incremental new customers: product of the treatment group        population and the incremental new customer rate.

Note that customer rates can be measured for different windows of time.For example, in certain embodiments, customer rate may be measured overa set time, such as for five days. The customer rate over the set timemay be used to predict a customer rate for a different time frame, suchas a thirty date customer rate, for optimization purposes. Allincremental customer rates cats be expressed as customer rates.

A general linear model 311 may determine differences in performancebetween a treated and control group in a marketing campaign based on theinput variables. Certain embodiments may use real time campaign data,audience data and attribution data as independent variables in a generallinear model. In certain embodiments, the model may use these variablesand weight them against each other to determine their effect on adependent variable (i.e., a projected cost per new customer rate for theentire campaign.) This output may be advertiser specific, but may befocused on return on investment for the marketing initiative inquestion. The outputs can be on a campaign or creative level, allowingoptimization of advertising spend and business decisions.

The general linear model may allow for prediction of a 30-60 dayattribution measurement in just days (compared to a traditional 30-60day window) upon reaching a statistically relevant volume. Astatistically relevant volume may depend on the advertising campaign inquestion, and may be based on a statistical significance test asdescribed above. The input 303 may be provided to or accessed by thegeneral linear model 311. The model 311 may determine one or moreinfluential factors in predicting total sales generated by a campaign.The factors may be weighted based on their expected influence on acampaign.

The model may predict online and offline campaign impact 313. Theresults may allow for reallocation of advertising spending to topperforming campaigns and audiences much faster than standard practices.Predictions 313 may project weekly cost per incremental customer acrossmultiple campaigns. Time periods for various embodiments may vary, andmay include real-time, near real-time, daily, weekly, monthly,quarterly, yearly, or other time periods. For example, the prediction313 may project customer acquisition cost for a customer on a weeklybasis giving the client the ability to shift advertising budget to thetop performing campaigns. In direct mail, customer acquisition costcalculations take up to six weeks to actualize.

Although the foregoing description is directed to the preferredembodiments of the invention, it is noted that other variations andmodifications will be apparent to those skilled in the art, and may bemade without departing from the spirit or scope of the invention.Moreover, features described in connection with one embodiment of theinvention may be used in conjunction with other embodiments, even if notexplicitly stated above.

1. A computerized method of predictive modeling of attribution, the computerized method comprising the steps of: selecting a first set of control electronic mail recipients; selecting a second set of treatment electronic mail recipients; transmitting, by a computing server, a first electronic mail message to first remote computing devices associated with the first set of control electronic mail recipients using a data communication network; transmitting, by the computing server, a second electronic mail message containing an element of a messaging campaign to second remote computing devices associated with the second treatment set of electronic mail recipients; receiving, by the computing server in real time over the communication network, first data indicating interactions of the first set of control electronic mail recipients with the first electronic mail message and the second data indicating interactions of the second set of treatment electronic mail recipients with the second electronic mail message; receiving, by the computing server, customer information including match information; generating, by the computing server, campaign-related attribution data using the customer information and a difference between the first data and the second data in response to one or both of the first data and the second data satisfying the match information; based on the difference between the first data and the second data, generating an incremental new customer rate of a campaign to which the campaign-related attribution data relates; generating a product of the second set of treatment electronic mail recipients and the incremental new customer rate to identify a set of new customers generated by the campaign; generating an incremental lift by comparing response rates from the first set of control electronic mail recipients and the second set of treatment electronic mail recipients; acquiring, by the computing server, real time campaign data, audience data, and attribution data from one or more databases, and adding the campaign-related attribution data to the attribution data, the real time campaign data including rate of change of metric rates; generating, by the computing server, a general linear model having one or more inputs from each of the real time campaign data, the audience data, and the attribution data as independent input variables in the general linear model; and predicting in real time, by the computing server, an online and offline campaign impact of the messaging campaign using the general linear model with the campaign-related attribution data as part of the independent input variables to determine differences in performance between the first set of electronic mail recipients and the second set of electronic mail recipients, the predicted campaign impact including data relating to at least one or more of the incremental new customer rate, the set of new customers, and the incremental lift; transmitting, by the computing server, a third electronic mail message based on the predicting.
 2. The method of claim 1, wherein selecting the first set of control electronic mail recipients and the second set of treatment electronic mail recipients includes using a sampling process such that attributes of the first set and the second set are proportional to each other.
 3. The method of claim 1, wherein the real time campaign data is further selected from the group consisting of: opens, clicks, landing page actions, complaints, unsubscribes, metrics rates, dates, times, and combinations thereof.
 4. The method of claim 1, wherein the audience profiles are selected from the group consisting of: demographics, geographic, online sales, offline sales, psychographic, purchase intent data, and combinations thereof.
 5. The method of claim 1, wherein the attribution data is selected from the group consisting of: advertiser customer data, treated prospects records, control prospects records, incremental customers, incremental customer rate, and combinations thereof.
 6. The method of claim 1, wherein the match information includes a time range for completion of an event with respect to the transmission of the second electronic mail message containing the element of the messaging campaign.
 7. The method of claim 1, wherein the method includes varying values of the real time campaign data, the audience data, and the attribution data that are input as independent input variables in the general linear model to determine weights indicating an importance of each of the real time campaign data, the audience data and the attribution data.
 8. The method of claim 7, wherein the general linear model determines influential factors.
 9. The method of claim 1, wherein the predicted online and offline campaign impact is determined on a weekly basis.
 10. A system for predictive modeling of online and offline attribution, the system comprising: one or more databases; system memory comprising instructions; and one or more processors to execute the instructions to perform operations comprising: selecting a first set of control electronic mail recipients; selecting a second set of treatment electronic mail recipients; transmitting a first electronic mail message to first remote computing devices associated with the first set of control electronic mail recipients using a data communication network; transmitting a second electronic mail message containing an element of a messaging campaign to second remote computing devices associated with the second set of treatment electronic mail recipients; receiving, in real time over the communication network, first data indicating interactions of the first set of control electronic mail recipients with the first electronic mail message and the second data indicating interactions of the second set of treatment electronic mail recipients with the second electronic mail message; receiving customer information including match information; generating campaign-related attribution data using the customer information and a difference between the first data and the second data in response to one or both of the first data and the second data satisfying the match information; acquiring real time campaign data, audience data, and attribution data from the one or more databases, and adding the campaign-related attribution data to the attribution data, the real time campaign data including rate of change of metric rates; generating a general linear model having one or more inputs from each of the real time campaign data, the audience data, and the attribution data as independent input variables in the general linear model; and predicting in real time an online and offline campaign impact of the messaging campaign using the general linear model with the campaign-related attribution data as part to the input independent input variables to determine differences in performance between the first set of electronic mail recipients and the second set of electronic mail recipients; transmitting, by the computing server, a third electronic mail message based on the predicting.
 11. The system of claim 10, wherein selecting the first set of control electronic mail recipients and the second set of treatment electronic mail recipients includes using a sampling process such that attributes of the first set and the second set are proportional to each other.
 12. The system of claim 10, wherein the real time campaign data is further selected from the group consisting of: opens, clicks, landing page actions, complaints, unsubscribes, metrics rates, dates, times, and combinations thereof.
 13. The system of claim 10, wherein the audience profiles are selected from the group consisting of: demographics, geographic, online sales, offline sales, psychographic, purchase intent data, and combinations thereof.
 14. The system of claim 10, wherein the attribution data is selected from the group consisting of: advertiser customer data, treated prospects records, control prospects records, incremental customers, incremental customer rate, and combinations thereof.
 15. The system of claim 10, wherein the match information includes a time range for completion of an event with respect to the transmission of the second electronic mail message containing the element of the messaging campaign.
 16. The system of claim 15, wherein the operations include varying values of the real time campaign data, the audience data, and the attribution data that are input as independent input variables in the general linear model to determine weights indicating an importance of each of the real time campaign data, the audience data and the attribution data.
 17. The system of claim 16, wherein the general linear model determines influential factors.
 18. The system of claim 10, wherein the predicted online and offline campaign impact is determined on a weekly basis. 